Towards a Very Fast Feedforward Multilayer Neural Networks Training Algorithm

نویسندگان

چکیده

Abstract ** This paper presents a novel fast algorithm for feedforward neural networks training. It is based on the Recursive Least Squares (RLS) method commonly used designing adaptive filters. Besides, it utilizes two techniques of linear algebra, namely orthogonal transformation method, called Givens Rotations (GR), and QR decomposition, creating GQR (symbolically we write GR + = GQR) procedure solving normal equations in weight update process. In this paper, approach to presented. The main idea revolves around reducing computational cost single rotation by eliminating square root calculation number multiplications. proposed modification scaled version rotations, denoted as SGQR. expected bring significant training time reduction comparing classic algorithm. begins with introduction description. Then, its usage decomposition discussed. section article network which rotations Next, experiment results are presented several benchmarks combined various topologies. shown that outperforms other methods, including well known Adam optimizer.

منابع مشابه

EAST: An Exponential Adaptive Skipping Training Algorithm for Multilayer Feedforward Neural Networks

Multilayer Feedforward Neural Network (MFNN) has been administered widely for solving a wide range of supervised pattern recognition tasks. The major problem in the MFNN training phase is its long training time especially when it is trained on very huge training datasets. In this accordance, an enhanced training algorithm called Exponential Adaptive Skipping Training (EAST) Algorithm is propose...

متن کامل

A Constructive Algorithm for Feedforward Neural Networks With Incremental Training

We develop, in this brief, a new constructive learning algorithm for feedforward neural networks. We employ an incremental training procedure where training patterns are learned one by one. Our algorithm starts with a single training pattern and a single hidden-layer neuron. During the course of neural network training, when the algorithm gets stuck in a local minimum, we will attempt to escape...

متن کامل

Evolutional Design and Training Algorithm for Feedforward Neural Networks

In pattern recognition using neural networks, it is very difficult for researchers or users to design optimal neural network architecture for a specific task. It is possible for any kinds of neural network architectures to obtain a certain measure of recognition ratio. It is, however, difficult to get an optimal neural network architecture for a specific task analytically in the recognition rat...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Journal of Artificial Intelligence and Soft Computing Research

سال: 2022

ISSN: ['2083-2567', '2449-6499']

DOI: https://doi.org/10.2478/jaiscr-2022-0012